1. Field of the Invention
The present invention generally concerns a client-server computer system, and a process, (1) for managing and maintaining, at one or more severs upon a worldwide computer communications network, a central or a distributed secure repository of digitalized three-dimensional [“3D”] models of 3D objects, and colors, textures, etc., related to these models and objects, and (2) for using client's(s') request(s) upon the network at the one or more servers these 3D models and related colors, textures, etc., for the generation of photorealistic perspective view and/or 3D images, and (3) for supplying the colored and/or textured image(s) so generated upon and across the same communications network to the requesting client(s), all the while prohibiting direct access to the 3D models and related colors, textures, etc. by the client(s).
In particular, the modeled 3D objects may be either real or virtual, and are most typically 3D objects shown for promotion as located in, surrounding, or in front of, either real or virtual 3D scenes. The photorealistic images generated of most typically non-fanciful objects and scenes that could be existing within the real world. These object and scene images are useful for (1) promoting or selling the depicted objects directly to a consumer, and/or (2) generating promotional images of the objects and/or (3) facilitating collaborative design.
The present invention further generally concerns a method of doing business upon a worldwide computer communications network by (1) managing and maintaining a secure central or distributed repository of high-resolution digital models and/or related colors, textures, etc. of objects and of scenes, and (2) providing of photorealistic image generation services based in part upon these models and related colors, textures, etc.
In particular, the (1) digitally modeled objects may be items such as architectural products, furniture and furnishings that are prospectively shown appropriately colored and textured as located within modeled scenes of a prospective purchaser's real property or equipment. In this eventuality, the (2) photorealistic image generation may be valuable to a prospective seller of architectural products, furniture and/or furnishings in making a sale of the photorealistically shown objects to the prospective purchaser.
2. Background
2.1 General Background of the Invention
The present invention will be seen to have both (i) business and (ii) technical aspects.
Technically, the present invention will be seen to broadly concern a secure digital model repository for use in, and with, the software-based system of the predecessor applications and patents for both (i) interactively producing, and (ii) rendering, across a digital communications network, photo-realistic composite images of interactively customized products in such surroundings as are customary to and specified by the viewer of the image, different viewers desiring and appreciating different surroundings.
Business-wise, the present invention will be seen to broadly concern a central role of this repository in profiting from the (1) management and maintenance of 3D digital models, and/or (2) a photorealistic image rendering system utilized for the purposes of promoting and selling such products, without requiring the distribution of proprietary high resolution 3D object models and/or related textures.
2.2 Previous Patents Directed to Image Generation in Interior or Exterior Design
U.S. Pat. No. 4,318,121 to Taite, et. al., for INTERIOR DECOR COMPOSITION AND DISPLAY SYSTEMS concerns display systems are of particular utility in assisting interior decorators and homemakers in selecting appropriate color and fabric combinations. In particular, the subject display systems provide a luminous image of an interior decor including components, such as furniture, drapes and carpets. The system picks up colors from different samples, each of which typically has an area smaller than the area of any displayed component. These picked-up colors are selectively filled in the displayed components when the image is displayed. Color arrangements are changed by substituting other picked-up colors in the display of particular components.
U.S. Pat. No. 4,931,929 to Sherman for DESIGN COMPONENT SELECTION COMPUTER WITH SPECIFICATION OF PRODUCT CHARACTERISTICS AND OF COLOR BY MACHINE READABLE DEVICE concerns a process for identification, description and display of design components, such as interior decoration products, selected by describing desired component characteristics to a digital computer. An image database is provided, preferably in the form of an addressable optical videodisk, each frame of which portrays an individual design component, and the images are displayed by operation of a computer-controlled archive system such as a random-access optical videodisk player. A characteristics database is provided to characterize each portrayed design component by a variety of product categories including at least color, price, manufacturer and image database address. The process obtains one or more desired component characteristics, examines the characteristics database to identify design components meeting the desired product characteristics, and displays the identified component images together with a complete record of characteristics for each product. The desired color characteristic can be obtained by selection from a text dictionary of color identification terms, by machine inspection of a machine-readable color comparison sample, or by optical spectrum analysis of a pre-existing product sample.
U.S. Pat. No. 5,255,352 to Falk for MAPPING OF TWO-DIMENSIONAL SURFACE DETAIL ON THREE-DIMENSIONAL SURFACES concerns a system and method for providing surface detail to mathematically-defined three-dimensional surfaces which preserves the specific dimensional integrity of the surface detail image being mapped in order to provide dimensionally correct surface detail. This is accomplished by performing the intermediate step of mapping the surface detail image to a two-dimensional flattened pattern piece representation of the surface and thereafter mapping this representation to the three-dimensional surface. The system and method of the present invention may be used as one function of a greater three-dimensional computer aided design system for any industry which utilizes flat sheets of material formed or assembled into a finished product, including textiles, apparel, footwear, upholstered furnishings, automotive or interior design.
U.S. Pat. No. 5,601,431 to Howard for an INTERIOR DESIGN SYSTEM AND METHOD concerns an interior design system comprising a display assembly having three flexible transparent overlays, a plain background sheet hinged to corresponding edges of the overlays, and a template sheet hinged to a free edge of the background sheet and having “scale” background and foreground grid sections. All this is for assisting in laying out interior room decorator's designs with a comprehensive catalog of design elements printed with scale designs of perspective furniture and accessory designs, and architectural elements for creating the room space to be decorated. The catalog pages have rows of printed scale designs covered with transparent plastic, with overlying, selectively removable and replaceable design elements. The design elements have designs that are printed to the same scale on carrier sheets of flexible “static-cling” material. These carrier sheets are rectangular sections that are cut from a printed sheet the same size and shape as the backing sheet. In this manner, the catalog pages provide marked storage positions for the design elements after use.
In addition, the backing sheets are marked with descriptive labels and special coded symbols in distinctive shapes and colors for assisting in selection and coordination of interior designs, and also are imprinted with generic labels for the designs. The method constitutes the steps of (i) providing the catalog of scale images and the background and foreground grid sections, (ii) labeling and coding the images for identification, coordination and storage, (iii) selecting candidate images from the catalog and transferring them to the grid sections, (iv) applying the images in the desired positions and combinations, thereby to create a decorator's design, plus, optionally, (v) applying colors as an overlay on the completed design for a more complete rendering.
U.S. Pat. No. 5,940,806 to Danial for a METHOD AND APPARATUS FOR RECEIVING, STORING AND PROVIDING DATA CORRESPONDING TO GEOMETRICALLY DESCRIBABLE RAW MATERIALS concerns an apparatus and method employing a programmable computer for providing a list of data items corresponding to geometrically describable materials based upon a profile specified by a user. The list of data items is stored in a data base. Each data item includes geometrically descriptive terms and trading parameters identifying characteristics of an item offered for sale by an advertiser. A profile is generated by a user which includes geometrically descriptive terms and trading parameters identifying characteristics of an item sought for purchase by the user. The profile identified by the user is compared to each data item stored in the data base and data items are identified which match the profile. The data item identified are provided to the user in accordance with the user's specific delivery instructions.
U.S. Pat. No. 6,005,969 to Thomas, et. al., concerns METHODS AND SYSTEMS FOR MANIPULATION OF IMAGES OF FLOOR COVERINGS OR OTHER FABRICS that enable a highly streamlined and efficient fabric or textile sampling and design process. The sampling and design process is claimed to be particularly valuable in the design and selection of floor coverings, wall coverings and other interior design treatments. A digital library of fabric models is created, preferably including digitized full-color images associated with digital representations of positions that are located within, and which characterize, the models. Via an application implemented according to conventional software methods and running on conventional hardware having high resolution graphics-processing capabilities, a user may navigate among the set of alternative models, and may modify the positions of the selected models to test out desired combinations of characteristics—such as poms or yarn ends, or models of floor coverings—and view the results in high resolution. A method is provided for substituting colors in digital images of photographic quality, while preserving their realism particularly in the vicinity of shadows. The resulting samples or designs can be stored and transmitted over a telecommunications network or by other means to a central facility that can either generate photographic-quality images of the samples, or can directly generate actual samples of the carpet or other material of interest.
2.3 A Specific Previous Interior Design System Operative on a Private Satellite Down-Link, Phone Line Up-Link Communications Network
The system and method of the present invention will be seen to involve interactive communications across a digital communications network for the purpose of producing images that typically include furniture. At least one previous upholstered furniture customization system involves a real-time network-based interactive system for support of visualizing upholstered furniture prior to placing an order for the upholstered furniture.
This existing interactive system of La-Z-Boy™ Incorporated, and possibly others, is available for use by sales associates in La-A-Boy™ furniture showrooms. The system permits prospective customers to preview La-Z-Boy furniture upholstered in La-Z-Boy offered fabrics. It requires a La-Z-Boy furniture item number, and a La-Z-Boy fabric number. The system then maps the fabric onto a static perspective view of the selected furniture item.
To the extent of depicting furniture, this previous system is similar to the system and method of the present invention. However, the system used by La-Z-Boy is considerably different from the system of the present invention as to each of, inter alia, (i) system interaction (rote versus sophisticated) with the targeted user (a furniture salesman versus an interior designer), (ii) how the furniture images are generated and presented (2D versus 3D, and fixedly versus being shown in context), (iii) the technical capabilities of the image generation (no provision for scaling, rotation and placement of upholstered furniture images into background scenes), and (iv) the business model supported (retail store furniture promotion vs. a professional-level interior design tool supporting furniture selection, placement and procurement as well as promotion.)
2.4 A Specific Interior Design System Operative on the Internet
The system and method of the present invention will be seen to involve communications across a digital communications network, to wit: the Internet. At least one interactive design system—evolved as a real-time network-based evolution of an older PC-based software program product—already (circa 2000) exists on the Internet. This existing system is superficially similar to the system and method of the present invention. However, the existing system is again considerably different from the system of the present invention as to each of, inter alia, (i) the targeted audience (amateurs versus professionals), (ii) how the images are generated and presented (2D versus 3D), and (iii) the business model supported (furniture and software promotion vs. professional-level design assistance, furniture promotion and procurement.)
The existing interactive system is the called the cMyVision Free Internet design software for use in interior, exterior, home and landscape improvement. The cMyVision software is a product, and cMyVision web site a service, of Visual Applications, Inc. (VisApp), headquartered in Kansas City, Mo. (cMyVision is a trademark of VisApp.) VisApp is a publisher of personal visualization technology, with a web site as of the year 2000.
In use of the cMyVision™ free Internet design software for interior, exterior, home and landscape improvement, a digital photo of a house, normally the client's own house, is initially required. Into this 2D scene a 2D object is inserted from the cMyVision library, or database. The system of the present invention will be seen to differ in that, from two or more 2D views, a 3D model of the client's 3D space, replete with dimensions, is first constructed. Then this 3D space is populated with 3D objects such as furniture. Accordingly, the cMyvision system operates to produce two-dimensional (2D) images while the system of the present invention, employing full three-dimensional (3D) models of both scenes and objects, produces full perspective scene views.
VisApp also sells a product called VisualPhile Personal Visualization Software, touted to support professional interior remodeling and design.
VisualPhile is personal computer software with the same features as cMyvision. It runs on a personal computer without an Internet connection, as its library is included with the program. It is primarily suited to providing photo-realistic visualizations of home remodeling as opposed to decorating projects. As in cMyVision, a certain degree of photo-realism stems from the use of photographs for both rooms and built-ins, appliances and furnishings.
However, there exist inherent limitations—arguably severe when applied to professional interior design—in both cMyVision sand VisualPhile and their use of digital photographs. These previous systems (i) are not based upon 3D coordinates (as will be seen to be used in the present invention), so furniture images have to be placed and scaled visually by the user; and (ii) the furniture objects are 2D images, so they cannot be rotated to match the angle of the room photograph (as will seen to be the case in the present invention). Therefore, in both cMyVision and VisualPhile, room photos must be taken from a specified perspective, as their furniture photographs are all taken from this angle. Moreover, (iii) because the furniture images are only photographs, and not 3D models, it is not possible to map fabrics on them. This last limitation cannot be mitigated and is highly significant in professional interior design applications.
2.5 Stand Alone Personal Computer Interior Design Programs
A number of software programs are commercially available which support interior design. These programs generally run on personal computers and allow the user to design rooms, place furniture objects into rooms, change wall colors and furniture fabrics, and display the customized furniture in the decorated rooms. Custom Home 3-D Design and Decor, published by Sierra On-line, Inc., 5 in 1 Home Design by Punch! Software LLC, and 3D Home Interiors by Broderbund all support most of these design-assistance and visualization objectives. In addition, numerous commercially-available programs support architectural rendering, including computer-aided design (CAD) programs like AutoCAD, which incorporate 3-D functionality and therefore have application to interior design.
Custom Home 3-D Design and Decor includes a program called Photo Home Interiors. This program, unlike the others, allows users to import photographs of actual rooms into the program, and then place furniture objects into the room images. The objects include 3-D furniture models, which can be rotated and scaled to fit the scene. However, the room images are not associated with a 3-D model of the room. In other words, while Photo Home Interiors does allow furniture objects to be shown as an overlay on top of room photographs, it lacks the capability of rendering furniture objects placed onto floor plans in photographically-derived scenes. Therefore, this program does not allow wall and floor coverings to be displayed in the proper perspective. In addition, the 3-D objects need to be visually scaled by the user to compensate for the perceived distance from the camera position. With this system, it is not possible to evaluate furniture objects of specific dimensions as to scale. For all of these reasons, is cumbersome to use on a personal computer, and would be even more so over a digital communications network.
Furthermore, the furniture objects are generic and therefore not available for purchase, no procurement features are available, the furniture and fabric database is very limited in quantity and quality. Finally, this program does not operate over a digital network.
As a result of such limitations, even this program has limited professional application. These limitations in the prior art are overcome in the present invention, which incorporates proprietary technology developed to support this highly desirable functionality.
2.6 Existing 3D Image Capture Technology
The present invention will be seen to beneficially employ in the generation of three-dimensional (3D) images of a human head a 3D image capture technology. Such a technology is commercially available, circa 2000, from Minolta Co., Ltd. in combination with MetaCreations, Inc., in the 3D 1500 camera product. The 3D 1500 is a complete hardware/software package that combines a quality Minolta digital camera with MetaCreation's MetaFlash 3D image production technology. It allows creation of high-quality 3D models in MetaCreation's MetaStream file format, which can be transmitted quickly over the network for further manipulation.
Another example of a three-dimensional (3D) image acquisition system for deriving 3D models of the human head is the ShapeSnatcher 3D-Toolkit from Eyetronics, Inc. After calibration, data can be captured within the time needed for a single photograph or frame. The 3D model is textured, with a perfect alignment between the shape and the texture. Skin color, marks or scars are visible as they would be on a photograph.
2.7 Relationship to the Predecessor Patent Applications
The present patent application is a continuation-in-part to U.S. patent application Ser. No. 10/081,841 filed on Feb. 20, 2002, which application itself is a continuation-in-part of U.S. patent application Ser. No. 09/643,507 filed on Aug. 22, 2000. The present application is related to these applications as being the substantial equivalent thereof, amplifying what is arguably only but a somewhat narrow point. Namely, the present invention and application particularly concerns the utilization of the previously-referenced network-linked rendering system to remotely render objects that are, in particular stored in a library that is secure. Using stand-in objects, as discussed in patent application Ser. No. 10/081,841, design professionals at client computers are able to “fine-tune” and present designs that incorporate objects stored securely in the library. The objects remain completely secure at one (i.e., centralized) or more (i.e., distributed) server computers, yet 2D perspective-view in-situ photorealistic renderings incorporating these objects can be both generated at the one or more sever computers, and subsequently remotely viewed at the one or more client computers.
This would allow, for example, an architect working at a sever computer to render a bathroom interior with a specific Kohler sink faucet, after Kohler had placed a high-resolution 3-D model of this faucet in the secure library(ies) of the one or more server computers. Clearly the security of this 3-D model, which is likely both expensive and proprietary, is maintained. This is useful to prevent unauthorized reverse engineering (copying) of the faucet from the 3-D model by a competitor.
The second predecessor application contains a variant teaching that, as applied to the secure library invention of the present application discussed above, permits a server-based rendering using very large (high-resolution) models and textures while permitting all such models and related information—like true textures and colors—to remain secure on the server. It would be very time-consuming and expensive (in a bandwidth sense) to download 50 MB of models for rendering on the client. Furthermore such downloads would compromise the security of the models, as they could then be stolen.
Given that relatively inexpensive high-performance computers are now readily available and in use by business, it is the security and data transmission speed issues that are the real competitive advantage of server-based rendering in accordance with the present invention in business applications. Also, keeping the large models and textures (and colors, etc.) on the server allows for relatively easy maintenance of the database. If the high-resolution models were distributed, updating them to current versions would become very difficult and expensive. The system of the present invention that combines client rendering of small stand-in models and textures to support scene specification and image preview, with server-based rendering of proprietary high-resolution models and textures is the best of both worlds.